Data Mining Tools Make Flights Safer, More Efficient 


NASA Technology 

When an aircraft experiences catastrophic failure, it’s not difficult to tell that something has gone wrong. 
But even if an airplane reaches its destination without an incident, it may still have bent or broken the 
boundaries of safe operating procedures. Figuring out what those boundaries should be — what “normal” 
operation should look like — and devising ways of effectively tracking them can be a tough task. 

Typically, this is accomplished using modeling and real-world tests, followed by equipping vehicles with 
a variety of sensors so that flight data can be recorded and analyzed. 

But an average commercial airplane generates a massive amount of data each flight: everything from 
instrument positions to sensors to voice recordings are collected and stored for later analysis. With so 
much data, there is no good way for problems to stand out unless a commercial company already knows 
what those problems look like. Says Captain Jeff Hamlett, the director of flight safety at Southwest 
Airlines Co., “We have mounds of data; the big request is always, ‘Tell me something I don’t know.’ We 
have to start with something specific, like an issue we discovered in a pilot report, and then we can search 
through the system and discover the breadth and depth of the issue.” 

Discovering these concerns and addressing them before they cause an accident is also one of NASA’s 
missions. The Aviation Safety Program under the Aeronautics Research Mission Directorate has many 
initiatives to enable safer air transportation systems. Among them, in the Intelligent Systems Division at 
Ames Research Center, is a small team of researchers who have been tasked with applying data mining 
algorithms to aviation safety problems. 

Technology Transfer 

NASA has lots of expertise in data mining — that is, creating tools to discover interesting patterns in large 
data sets. But in order to ensure its techniques can make a real impact on flight safety and efficiency, 
Ames has sought the help of commercial partners, among them Dallas-based Southwest. 

Says Nikunj Oza, a researcher at Ames and leader of the data mining team, “We’ve made partnerships 
with Southwest and others, who then make their data accessible to us to facilitate our development. In 
return, they get certain insights into their data, and they can give us feedback on how well our algorithms 
are working, as well as what adjustments might make them more accurate or user friendly.” 

For Hamlett, the value of a NASA-Southwest partnership had become apparent in 2003, when Southwest 
identified a higher rate of steep approaches in the flight data than was being reported by its pilots. “We 
turned to our pilot self-reporting program, expecting to find lots of reporting about unstable approaches, 
but we couldn’t match the number that we were seeing in our data,” he says. 

Hamlett tried instead searching by phrases within pilot reports, using a NASA-developed tool called 
Perilog. “It turns out that when you read some of the elements of the report,” he says, “they are describing 
an unstable approach — saying things like ‘we were struggling to get the airplane configured’ or ‘we were 


too fast’ — and using all the words that describe an unstable approach, even if they hadn’t reported it as 
such.” Southwest used reported information to identify and communicate certain issues to air traffic 
controllers, making them aware, for example, of how certain instructions impacted an airplane’s 
operations. By collaborating with them, Southwest has since seen a steady improvement in the quality of 
approaches. 

“That’s just one example of how NASA provided us a tool that helped us look at our data in a way we 
never thought about before,” says Hamlett. 

Another collaborative effort between Southwest and the Human Factors Division at Ames resulted in an 
improved, rewritten set of operating procedures for the airline in 2004. Says Hamlett, “They played a big 
part in making sure that we get a good process for reviewing all of our normal operations.” And from that 
work, Hamlett made the acquaintance of Ashok Srivastava, then head of the data mining team in the 
Aviation Safety Program at NASA. 

Following year-long discussions between Hamlett and Srivastava, in 201 1 NASA and Southwest signed a 
Space Act Agreement to share algorithms and flight data. The arrangement would help the Ames team 
ensure that their research was suitable for real-world applications, and it would help Southwest to make 
the most out of its large and growing body of data. 

Benefits 

As Oza puts it, NASA’s algorithms — including the Multivariate Time Series (MTS) search and Virtual 
Sensors — specialize in “letting the data speak for itself by finding unusual flights and candidate anomalies, 
without having any preconceived notion of what is normal or abnormal. The drawback is that sometimes you 
find statistical anomalies that are not safety concerns, but the benefit is that sometimes you’ll find anomalies 
that you weren’t seeing before that turn out to be safety concerns or have other operational significance such as 
excessive fuel use.” 

One such revelation came when Southwest began tracking planes in an effort to boost fuel efficiency. Says 
Hamlett, “What we were trying to do was look at the perfonnance of aircraft between certain cities, to see if 
there was a particular plane or city pair that was off normal and causing us to bum too much fuel.” What 
Southwest discovered instead was an unusual signature with one particular plane, which when investigated 
turned out to be caused by inaccurate sensors. “It wasn’t what we set out to do, but it turned into a success for 
us,” says Hamlett. 

Like many airlines, Southwest has traditionally looked for performance issues in its data using exceedances 
checked against a model. During arrival, for instance, the plane might record an exceedance if it is travelling 
faster than 250 knots while its altitude is less than 10,000 feet. Each morning, the company looks at a report of 
all the exceedances that took place the previous day and decides what action to take in the case of undesirable 
trends. 

Using NASA’s tools, Hamlett says Southwest can now query the data itself to figure out what normal 
operations really look like. “Just because we expect an airplane to be on speed at 1,000 feet, on glide path, on 
course, doesn’t mean that’s what normally happens. I think what we got out of this technology is the ability to 


ask, ‘What is normal?’ Because that turns out to be really powerful, and we can say ‘OK, we need to make this 
correction in our training,’ or ‘Maybe we need to adjust our concept of what the ideal is.’” 

Oza is glad to see that NASA’s work is making an impact on commercial flight safety. “If you go into a dark 
room with a flashlight, your beam only hits a small area, and you don’t know what’s in the space that’s dark. 
We’re essentially going in with a broader beam, if not a floodlight. What you find may or may not turn out to 
be problematic — but at least you now know what’s there.” 


